Low-Rank Modular Reinforcement Learning via Muscle Synergy

Modular Reinforcement Learning (RL) decentralizes the control of multi-joint robots by learning policies for each actuator. Previous work on modular RL has proven its ability to control morphologically different agents with a shared actuator policy. However, with the increase in the Degree of Freedom (DoF) of robots, training a morphology-generalizable modular controller becomes exponentially difficult. Motivated by the way the human central nervous system controls numerous muscles, we propose a Synergy-Oriented LeARning (SOLAR) framework that exploits the redundant nature of DoF in robot control. Actuators are grouped into synergies by an unsupervised learning method, and a synergy action is learned to control multiple actuators in synchrony. In this way, we achieve a low-rank control at the synergy level. We extensively evaluate our method on a variety of robot morphologies, and the results show its superior efficiency and generalizability, especially on robots with a large DoF like Humanoids++ and UNIMALs.Comment: 36th Conference on Neural Information Processing Systems (NeurIPS 2022

Knowledge orchestration and digital innovation networks: insights from the Chinese context

As digital innovation increasingly pushes heterogeneous actors to connect with each other across multiple organizational and community boundaries, a doubly distributed innovation network may emerge, leading to the knowledge being too fragmented and heterogeneous. Facing this problem, I place an emphasis on material artefacts and social network structures in the cultural context of Chinese digital innovators. On the one hand, as innovation is increasingly mediated by material artefacts, I focus on epistemic objects and activity objects, which are able to motivate the process of innovation. On the other hand, as innovation transforms the network actors’ social space, I focus on the role of “guanxi” (i.e. a system of influential relationships in Chinese culture) and structural holes (i.e. the absence of a connection between two contacts) in digital innovation networks. At the same time, as the literature recognizes knowledge orchestration as a useful starting point to address the knowledge fragmentation and heterogeneity, I identify five activities as knowledge orchestration: knowledge mobilization, knowledge coordination, knowledge sharing, knowledge acquisition and knowledge integration. As traditional tools used to support knowledge management can no longer handle the fragmented and heterogeneous knowledge, there is limited studies contributing to our understanding of how the Chinese innovators use objects and social network structures to orchestrate knowledge in their innovation networks. With these paucities of research in mind, this thesis explores how the material objects and the social network structures orchestrate knowledge for coordinating the fragmented and heterogeneous knowledge in Chinese digital innovation networks. From the perspective of material artefacts, my first study explores how epistemic objects affect the acquisition, integration and sharing of knowledge among collaborative organizations during their IT innovation alliances. My second study explores how activity objects affect the sharing, acquisition and integration of knowledge for crowdsourced digital innovation. From a social perspective, my third study explores how guanxi and structural holes affect the mobilization and coordination of knowledge among Chinese digital entrepreneurs in their innovation networks. Following the three studies, I show my key contributions, and discuss my theoretical and practical implications

Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks

We analytically investigate how over-parameterization of models in randomized machine learning algorithms impacts the information leakage about their training data. Specifically, we prove a privacy bound for the KL divergence between model distributions on worst-case neighboring datasets, and explore its dependence on the initialization, width, and depth of fully connected neural networks. We find that this KL privacy bound is largely determined by the expected squared gradient norm relative to model parameters during training. Notably, for the special setting of linearized network, our analysis indicates that the squared gradient norm (and therefore the escalation of privacy loss) is tied directly to the per-layer variance of the initialization distribution. By using this analysis, we demonstrate that privacy bound improves with increasing depth under certain initializations (LeCun and Xavier), while degrades with increasing depth under other initializations (He and NTK). Our work reveals a complex interplay between privacy and depth that depends on the chosen initialization distribution. We further prove excess empirical risk bounds under a fixed KL privacy budget, and show that the interplay between privacy utility trade-off and depth is similarly affected by the initialization

Empowering Graph Representation Learning with Test-Time Graph Transformation

As powerful tools for representation learning on graphs, graph neural networks (GNNs) have facilitated various applications from drug discovery to recommender systems. Nevertheless, the effectiveness of GNNs is immensely challenged by issues related to data quality, such as distribution shift, abnormal features and adversarial attacks. Recent efforts have been made on tackling these issues from a modeling perspective which requires additional cost of changing model architectures or re-training model parameters. In this work, we provide a data-centric view to tackle these issues and propose a graph transformation framework named GTrans which adapts and refines graph data at test time to achieve better performance. We provide theoretical analysis on the design of the framework and discuss why adapting graph data works better than adapting the model. Extensive experiments have demonstrated the effectiveness of GTrans on three distinct scenarios for eight benchmark datasets where suboptimal data is presented. Remarkably, GTrans performs the best in most cases with improvements up to 2.8%, 8.2% and 3.8% over the best baselines on three experimental settings

Single-Cell Multimodal Prediction via Transformers

The recent development of multimodal single-cell technology has made the possibility of acquiring multiple omics data from individual cells, thereby enabling a deeper understanding of cellular states and dynamics. Nevertheless, the proliferation of multimodal single-cell data also introduces tremendous challenges in modeling the complex interactions among different modalities. The recently advanced methods focus on constructing static interaction graphs and applying graph neural networks (GNNs) to learn from multimodal data. However, such static graphs can be suboptimal as they do not take advantage of the downstream task information; meanwhile GNNs also have some inherent limitations when deeply stacking GNN layers. To tackle these issues, in this work, we investigate how to leverage transformers for multimodal single-cell data in an end-to-end manner while exploiting downstream task information. In particular, we propose a scMoFormer framework which can readily incorporate external domain knowledge and model the interactions within each modality and cross modalities. Extensive experiments demonstrate that scMoFormer achieves superior performance on various benchmark datasets. Remarkably, scMoFormer won a Kaggle silver medal with the rank of 24/1221 (Top 2%) without ensemble in a NeurIPS 2022 competition. Our implementation is publicly available at Github.Comment: CIKM 202

VQ-NeRF: Vector Quantization Enhances Implicit Neural Representations

Recent advancements in implicit neural representations have contributed to high-fidelity surface reconstruction and photorealistic novel view synthesis. However, the computational complexity inherent in these methodologies presents a substantial impediment, constraining the attainable frame rates and resolutions in practical applications. In response to this predicament, we propose VQ-NeRF, an effective and efficient pipeline for enhancing implicit neural representations via vector quantization. The essence of our method involves reducing the sampling space of NeRF to a lower resolution and subsequently reinstating it to the original size utilizing a pre-trained VAE decoder, thereby effectively mitigating the sampling time bottleneck encountered during rendering. Although the codebook furnishes representative features, reconstructing fine texture details of the scene remains challenging due to high compression rates. To overcome this constraint, we design an innovative multi-scale NeRF sampling scheme that concurrently optimizes the NeRF model at both compressed and original scales to enhance the network's ability to preserve fine details. Furthermore, we incorporate a semantic loss function to improve the geometric fidelity and semantic coherence of our 3D reconstructions. Extensive experiments demonstrate the effectiveness of our model in achieving the optimal trade-off between rendering quality and efficiency. Evaluation on the DTU, BlendMVS, and H3DS datasets confirms the superior performance of our approach.Comment: Submitted to the 38th Annual AAAI Conference on Artificial Intelligenc